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Abstract - An autonomic system provides self-adaptive ability 
that enables system to dynamically adjust its behavior on 
environmental changes or system failure. Fundamental 
process of adaptive behavior in an autonomic system is consist 
of monitoring system or/and environment information, 
analyzing monitored information, planning adaptation policy 
and executing selected policy. Evaluating system utility is 
one of a significant part among them. We propose a novel 
approach on evaluating autonomic system at runtime. Our 
proposed method takes advantage of a goal model that has 
been widely used at requirement elicitation phase to capture 
system requirements. We suggest the state-based goal model 
that is dynamically activated as the system state changes. In 
addition, we defined type of constraints that can be used to 
evaluate goal satisfaction level. We implemented a prototype 
of autonomic computing software engine to verity our proposed 
method. We simulated the behavior of the autonomic 
computing engine with the home surveillance robot scenario 
and observed the validity of our proposed method 

Index Terms — autonomic computing, adaptive systems, goal 
model, embedded system 

I. Introduction 

As computing systems runtime environment is becoming 
extremely complex, unpredictable situations those were 
unforeseen at system design phase frequently occurs at 
runtime. To cope with such situations after system 
deployment, enormous cost might be required, especially for 
the large scale embedded systems such as a cyber physical 
system(CPS). Autonomic computing has been proposed to 
deal with such issue in pursuit of developing software that 
adapt to its own situation and minimize the occurrence of the 
system failure without human intervention. Such autonomic 
systems usually consist of system properties that deliver a 
monitoring, analysis, planning and execution feedback loop. 
Evaluation of the autonomic system is one of a crucial part 
among various research area related to autonomic computing. 
If an autonomic system provides a wrong evaluation of the 
adaptation result, such system might continuously make a 
wrong decision. Despite such importance, there are not 
enough leading research and actual practice. In this paper, 
we propose a novel approach on evaluating autonomic system 
at runtime. Our proposed method exploits a goal model that 
has been widely used at requirement elicitation phase to 
capture system requirements. To utilize a goal model on runtime 
evaluation of the autonomic system, we defined constraints 
that can be regarded as a runtime attribute of the goal model. 
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Furthermore, to reduce the message transmission overhead, 
our goal model is dynamically activated depending on the 
system state. 

This paper is organized as follows: the next section 
provides a brief introduction of autonomic computing and 
other related researches. Section III presents our self-adaptive 
autonomic system framework and autonomic systems runtime 
evaluation methodology. Section c! presents the 
implementation details of our prototype autonomic system 
and evaluation results. In section d!, we conclude our work 
with future research directives. 

II. Backgrounds 

A. Autonomic Computing 

Autonomic computing (AC) is a computing paradigm 
proposed by IBM[5] to solve the problems associated with 
the complex computing systems and unpredictable 
operational environment. In conventional computing 
paradigm, software engineers design, implement and test the 
software assuming that the system will be operated under 
predictable runtime environment predefined at requirement 
elicitation process. As system execution environment have 
become extremely complex, it is impossible to predict every 
situation a system might undergo. Autonomic computing 
introduced the concept of the human autonomic nervous 
system, which controls vital bodily function without human 
intervention. Autonomic computing allows such 
functionalities to the system by embedding additional 
infrastructure in the system. The core aspect of autonomic 
computing systems is as below: 

Self-configuration - Automatically re-configuring system 
components or integrating new components as system policy. 

Self-optimization - Continuously attempt to improve 
system performance and efficiency seamlessly. 

Self-healing - Automatically detects, localize system failure 
and recovers software and hardware problems 

Self-protection - System defends itself from malicious 
attacks or failures. 

B. Utility-based Self-adaptive System Evaluation 

Self-adaptive systems change system behavior or 
structures without human operator involvement. Therefore, 
deciding when to execute adaptive policies and evaluating 
how effectively applied policies actually improved system 
utilities are crucial. Kephart[9] suggested a system utility 
evaluation method for self-adaptive system. The author 
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separated self-adaptive system utility as service level utility 
and resource level utility. The method calculates every 
possible resource level utility to find the best amount of 
resource to be allocated. D. Garlan[10][l 1] also proposed 
similar approach on calculating system utility in Rainbow 
self-adaptive software architecture. Such methods were 
applied to a large-scale data center and web server system 
respectively to show validity. Both of aforementioned 
methods adopt limited number of metric to calculate overall 
system utility such as response time. However, such method 
is limited to a system with multiple objectives such as a 
robotics system, since a robotics system generally offers 
various functions. 

C. Goal Oriented Requirement Engineering 

A goal model has been used to capture system 
requirements for traditional systems on requirement 
elicitation phase. Goals tend to reflect what the system is 
supposed to do, and it has been regarded as a natural way to 
model system requirements. 

The Knowledge Acquisition in automated 
Specification(KAOS) framework is a well-known approach 
for modeling system requirements using a goal model[12]. 
The KAOS model is used at requirement elicitation phase to 
capture system goals. System goals are specified at high- 
level goals, and each of them is then decomposed into sub- 
goals that elaborate how the goal is achieved. The i* frame- 
work is another goal-oriented requirement engineering ap- 
proach that focus on the interaction between actors. There 
are several recent researches that extend existing approaches 
to deal with uncertainty or inconsistency. However, as far as 
we know there is no approach that takes advantage of a goal 
model to evaluate autonomic systems behavior and the re- 
sult of adaptation at runtime. 

III. Proposed approach 

In this section, we first give a brief overview of our 
autonomic computing software framework. Then the detailed 
explanation about how the autonomic computing engine's 
analysis and evaluation are performed will be followed. 

A. Overall Self-adaptive System Framework 

Fig. 1 shows an overall structure of our autonomic 
computing software engine. It is an improved version of our 
previous work[3]. We originally exploited rule -based tables 
as knowledge models, but we replaced these with tree 
structured goal models, and a fault tree. 

The autonomic computing engine in Fig. 1 interacts with 
the managed element, namely the actual running system, 
through the sensor and the effecter. The autonomic 
computing engine continuously checks the system status at 
runtime, and decides whether adaptation policies are required. 
A detailed functionality of each module in the autonomic 
computing engine is as follows: 

Monitoring module - The monitoring module requests 
required data to the managed element and receives the data 
through the sensor. Monitored data are then reported to the 
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analysis module to check goal violation. 
Analysis module - The analysis module examines received 
data based on the analysis model(goal model) in the knowledge 
base. Analysis model actually computes goal satisfaction 
level at runtime. The analysis module sends goal violation 
signal to the diagnosis module when goal satisfaction level 
is below the predefined threshold. A detailed process of 
calculating goal satisfaction level is described in section III - 
B. 

Diagnosis module - The diagnosis module receives goal 
violation information from the analysis module. Failure 
information from the analysis module is then mapped into 
the top event of the diagnosis model, which is represented 
as a fault tree in our framework. Fault tree is a graph 
representing the combination of the fault events that can 
derive specific failure of a system. Conventional fault tree is 
used at system design phase to analyze system reliability, 
but we utilize a fault tree at runtime to localize the root cause 
of the failure. Since the detailed mechanism of the runtime 
fault tree is out of this papers scope, we do not describe it in 
this paper. 

Planning Module - The planning module receives root cause 
information from the diagnosis module. The planning module 
choose corresponding plan among several alternative plans 
and pass the chosen policy id to the execution module. The 
execution module then sends policy id to the managed 
elements through the effecter. After policy execution, the 
evaluation module receives constraint information again from 
the managed elements and sends calculated goal satisfaction 
level to the planning module for feedback. 
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Figure 1 Overall autonomic computing engine architecture 

B. Runtime evaluation 

To evaluate system status at runtime, we exploit a goal 
model. By using a goal model, a system can trace current 
system status so that deciding which data to monitor be- 
comes easier. In case of a simple system such as a web server, 
only limited metrics such as response time and server cost 
are enough to monitor the system performance. On the other 
hand, a system that supports complex services such as ro- 
botic systems, an evaluation metric should be changed con- 
sidering current system status or objectives that the system 
is supposed to achieve. The basic structure of our goal model 
is similar with the conventional goal model such as the one 
used in KAOS. Fig. 2(a) shows an example goal model of our 
approach. The model represents the basic requirements of a 
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home surveillance robot that is capable of move to the desti- 
nation, avoiding obstacles, taking a picture with embedded 
camera, reporting environment information such as humidity 
and temperature, and constant battery checking. Upper level 
goals(parent node) are decomposed into lower level goals(child 
node) and each child has a weight depending on the contri- 
bution level to a parent goal. Each weight can be gained by 
using weight elicitation methods in the value tree analysis 
[14]. 

At runtime, an entire goal model shown in fig. 2(a) is not 
fully maintained. Instead, the goals those are related to current 
system state are activated at runtime. For example, when a 
robot's current goal is to move to some location (Move state), 
activated goals at this time will be GO, Gl, G4, G5, G6 and G12, 
as shown in fig. 2(b). The robot is supposed to avoid 
obstacles in the moving path, so when the robot finds an 
obstacle, the system state will be changed from Move state 
to Avoid Obstacles state. Therefore, the goals corresponding 
to Avoid Obstacles state will be automatically activated as 
the system state changes (fig. 2(c)). Each leaf goal has one or 
more constraints that can be calculated at runtime. There are 
two kinds of constraints in our model; maintenance 
constraints(MC) and operation constrains(OC). 




Figure 2 Example goal models 



MC represents a desire to keep a condition true over entire 
operation time. For example, when a robot is not supposed to 
stop during the Move state, a MC could be keep the speed 
over 0.0(or some other threshold value). MCs are periodically 
monitored during runtime at certain time interval, and the 
analysis module captures the goal violation as soon as MCs 
are violated. OCs represents the constraints those can be 
evaluated after execution of the goal. For example, when a 
requirement for a robot is to reach to a certain destination, 
OC becomes elapsed time t to reach to the destination. On 
evaluating OCs, we adopted fuzzy concept to deal with the 
ambiguity around the constraint boundary. For instance, if 
aforementioned OC is set to return 0.0 of satisfaction level 
after time t, the result will be 0.0 no matter how close the 
robot accomplished the goal to the requirement, and 
unnecessary adaptation process will continuously take place. 
Adopting fuzzy concept to a goal model is initially proposed 

©2012 ACEEE 
DOI:01.DNS.03.02.91 



by Baresi[8], but they only applied it to non-functional 
requirement in the goal model. In our perspective, providing 
flexibility on evaluating functional requirements is also 
necessary in many cases, so we extended fuzzy concept to 
every leaf goals. To measure the satisfaction level of operation 
constraints, a membership function should be set. Fiq. 3 show 
examples of membership functions. 

The evaluation module in fig. 1 calculates the total goal 
satisfaction level. Each activated leaf goal's achievement 
level(LGA) is measured with the equation below: 
□ 

LGA(fJ t ) = MC k ■ Wj ■ f w (OCJ (1] 

i = L 

where G, denotes k th goal in the goal model and MC,, OC, is 

k k ki 

the satisfaction level of the maintenance constraint and the 
operation constrain respectively. f u is i th operation 
constraint's membership function, where n is the number of 
the operation constrains, so the operation constraints are 
weighted sum of each constraint. After calculating LGAs, 
overall achievement level of the current goal model is also 
calculated with weighted sum method. If a goal node has one 
or more leaf goals, equation(2) is applied, otherwise equation 
(3) is recursively calculated until it reach to the root goal. 

GA(Gjk) - Ytx w i m tGi4(Gj (2) 
GA(G k ) = ^ =1 w r GA(G t ) (3) 

IV. Evaluation 

A. Scenario 

Due to the lack of freely available open source autonomic 
systems, we implemented a prototype of an autonomic 
computing engine to validate the feasibility of our proposed 
method(fig. 4). Our prototype is based on the home 
surveillance robot, which has six available states; start, idle, 
move, avoid obstacle, recharge, environment information 
report, shoot pictures and end. The autonomic computing 
engine was developed in Java using Eclipse SDK, along with 
SQLiteJDBC a Java driver for SQLite. In this prototype system, 
the autonomic computing engine is not embedded in the 
managed element, so we took advantage of TCP/IP protocol 
for the communication between the managed element and 
the autonomic computing engine. This protocol can be 
replaced with inter-process communication(IPC) when the 
autonomic engine is embedded into the managed element. 
To demonstrate the home surveillance robot, we implemented 
a simulated home environment. The robot consists of eleven 
sensors that can be monitored by autonomic computing 
engine. Our first experiment was set to evaluate whether the 
system accurately evaluates the system utility and detects 
goal violation. We simulated the robots moving behavior 
that belongs to the move state. The robot continuously sent 
constraint information(current velocity in this case) to the 
autonomic computing engine when moving from the starting 
point to the destination point. When the robot reached to 
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the destination, it sent operation constraint information to 
the autonomic computing engine to calculate goal 
satisfaction level. To validate our state-based goal model in 
terms of efficiency, we also simulated each available state 
and counted the number of messages sent between the home 
surveillance robot and the autonomic computing engine. For 
the counterpart, we counted the number of messages when 
the full goal model is kept during operation. 




B. Experimental Result 

Fig. 6 shows the operation log message of the robot and 
the autonomic computing engine. In fig. 6(a), the surveillance 
robot continuously sent maintenance constraint information 
to the autonomic computing engine during move operation. 
As soon as the robot arrived at the destination, operation 
constraint information(constraint ID(CID2-elapsed time)) was 
sent to the autonomic computing engine. In fig. 6(b), we 
simulated that the robot was stalled on the middle of path, 
and CID1 (current velocity) became 0.0, so the autonomic 
computing engine instantly detected the goal violation. In 
fig. 6(c), the robots move behavior ended without any goal 
violation, so the autonomic computing engine received the 
operation constraint information from the robot and calculated 
goal satisfaction level based on current active goal tree(fig. 
2(b)). We set the elapsed time to 7.0 seconds, where the 
designated constraint on goal model to finish move operation 
was 6.0seconds, therefore the membership function shown 
in fig. 3(a) returned 0.714 of satisfaction level. 

In our second set of experiments, we calculated total 
number of messages sent between the autonomic computing 
engine and the home surveillance robot to measure the cost 
of monitoring. We simulated the autonomic computing 
engine's move, obstacle avoid, environment information 
report, shoot pictures and charge battery state with state- 
based goal model that we propose. 



Figure 4 Class diagram of the autonomic computing engine 
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Table 1 Number of messages between autonomic computing engine and the 
home surveillance robot 
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At the same time, we simulated the autonomic behavior with 
a goal model that fully maintains state information. As 
expected, the result shown in table. 2 indicates that for any 
possible states, our proposed method reduced the number 
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>) (c) 

runtime evaluation 

of messages. On average, the number of messages passed 
was reduced 20% than its counterpart. This is due to our 
goal model that dynamically activates goals as system status 
changes. Since unnecessary goals in current state are 
excluded, the number of messages could be reduced. 

Conclusions 

We have proposed a runtime evaluation method for the 
autonomous system. Our evaluation method takes advantage 
of stated-based goal model that dynamically activates the 
goal model depending on the system state. Through a 
simulated prototype system, we have observed that our 
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proposed method can properly evaluate system utility and 
detect system failures. In addition, our experiment has shown 
that the monitoring overhead from message passing was 
reduced by 20% with our state-based goal model. 

For the future work, we are investigating ways to 
dynamically changing weights in the goal model, depending 
on the contextual situation. This would provide the autonomic 
system a more robust way to evaluate system utility in various 
environments. 
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